Debiasing convex regularized estimators and interval estimation in linear models

New upper bounds are developed for the $L_2$ distance between $\xi/\text{Var}[\xi]^{1/2}$ and linear quadratic functions of $z\sim N(0,I_n)$ random variables form $\xi=bz^\top f(z) - \text{div} f(z)$. The approximation yields a central limit theorem when squared norm $f(z)$ dominates Frobenius $\nabla f(z)$ in expectation. Applications this normal given asymptotic normality de-biased estimators regression with correlated design convex penalty regime $p/n \le \gamma$ constant $\gamma\in(0,{\infty})$. For estimation $\langle a_0,\beta\rangle$ unknown coefficient vector $\beta$, analysis leads to estimate most normalized directions $a_0$, where ``most'' is quantified precise sense. This holds any if $\gamma<1$ strongly $\gamma\ge 1$. In particular needs not be separable or permutation invariant. By allowing arbitrary regularizers, results vastly broaden scope applicability de-biasing methodologies obtain confidence intervals high-dimensions. absence strong convexity $p>n$, obtained Lasso group under additional conditions. general penalties, our also provides prediction error independent interest.